BY-COVID - WP5 - Baseline Use Case: COVID-19 vaccine effectiveness assessment

Survival plot

We estimate the survival function using the Kaplan-Meier estimator and represent this function visually using a Kaplan-Meier curve, showing the probability of not getting infected by SARS-CoV-2 at a certain time after onset of follow-up. The survival function is estimated for the control and intervention group.

The cumulative incidence of the event (SARS-CoV-2 infection) was additionally plotted.

Survival (time-to-event)

The probability of not getting infected by SARS-CoV-2 beyond a certain time after onset of follow-up (survival function, estimated using the Kaplan-Meier estimator) is reported for different periods.

Strata	Time	Number at risk	Cumulative sum of number of events	Cumulative sum of number censored	Survival	Std. error	Cumulative hazard	Std. error cumulative hazard
Not fully vaccinated	0	7757	0	0	1.0000	0.00000	0.0000	0.00000
Not fully vaccinated	100	2629	55	5092	0.9869	0.00186	0.0132	0.00188
Not fully vaccinated	200	861	193	6721	0.9053	0.00716	0.0995	0.00790
Not fully vaccinated	300	595	301	6861	0.7684	0.01360	0.2632	0.01767
Not fully vaccinated	400	461	324	6984	0.7383	0.01444	0.3030	0.01953
Not fully vaccinated	500	84	325	7350	0.7364	0.01453	0.3056	0.01970
Fully vaccinated	0	7757	0	0	1.0000	0.00000	0.0000	0.00000
Fully vaccinated	100	2628	60	5086	0.9857	0.00193	0.0144	0.00196
Fully vaccinated	200	841	249	6684	0.8737	0.00822	0.1349	0.00939
Fully vaccinated	300	603	334	6820	0.7676	0.01300	0.2642	0.01691
Fully vaccinated	400	482	352	6931	0.7447	0.01369	0.2945	0.01836
Fully vaccinated	500	89	352	7318	0.7447	0.01369	0.2945	0.01836

Median survival time

The median survival time is the time corresponding to a probability of not obtaining a SARS-CoV-2 infection probability of 0.5. (if NA, the probability of not obtaining a SARS-CoV-2 infection did not drop below 50%)

Characteristic	Median survival (95% CI)
fully_vaccinated_bl
FALSE	— (—, —)
TRUE	— (—, —)

Cox regression and estimation of the average treatment effect

A Cox regression model was built to examine the relationship between the distribution of the probability of not obtaining a SARS-CoV-2 infection (survival distribution) and completing a primary vaccination schedule (covariate). The Cox proportional hazards regression model was fitted with ‘fully_vaccinated_bl’ as a covariate and accounts for clustering within individuals (as one individual can be re-sampled as control).

A hazard ratio (HR) is computed for the covariate ‘fully_vaccinated_bl’. A hazard can be interpreted as the instantaneous rate of SARS-CoV-2 infections in individuals that are at risk for obtaining an infection (Cox proportional hazards regression assumes stable proportional hazards over time). A HR < 1 indicates reduced hazard of SARS-CoV-2 infection when having completed a primary vaccination schedule whereas a HR > 1 indicates an increased hazard of SARS-CoV-2 infection.

	Parameter estimate	SE coefficient	Robust SE coefficient	P-value	Hazard Ratio (HR) (95% CI for HR)
fully_vaccinated_blTRUE	0.083	0.077	0.1	0.41	1.086 (0.873, 1.3)

The overall significance of the model is tested.

	Test statistic	Df	P-value
Likelihood ratio test	1.1538840	1	0.2827376
Wald test	0.6800000	1	0.4103826
Score (logrank) test	1.1536006	1	0.2827967
Robust score test	0.6978656	1	0.4035018

Proportional hazards during the study period might be unlikely. As such, the RMST and RMTL ratios are additionally calculated, providing an alternative estimate for the the Average Treatment Effect (ATE), without requiring the proportional hazards assumption to be met.

Arm	Measure	Estimate	SE	CI.lower	CI.upper
fully_vaccinated_bl==FALSE	RMST	326.369	2.049	322.354	330.384
fully_vaccinated_bl==TRUE	RMST	324.198	2.045	320.189	328.207
fully_vaccinated_bl==FALSE	RMTL	38.631	2.049	34.616	42.646
fully_vaccinated_bl==TRUE	RMTL	40.802	2.045	36.793	44.811

Measure	Estimate	CI.lower	CI.upper	p_value
RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE)	-2.171	-7.845	3.503	0.453
RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE)	0.993	0.976	1.011	0.453
RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE)	1.056	0.915	1.219	0.454

Survival plot

The cumulative incidence of the event (SARS-CoV-2 infection) was additionally plotted within subsets defined by the vaccination schedule.

Cox regression and estimation of the average treatment effect

A Cox regression model was built to examine the relationship between the distribution of the probability of not obtaining a SARS-CoV-2 infection (survival distribution) and completing a primary vaccination schedule (covariate), and whether this differs according to the administered vaccination schedule. A stratified Cox proportional hazards regression model was fitted with ‘fully_vaccinated_bl’ as a covariate, ‘vaccination_schedule_cd’ as a stratification factor, and accounting for clustering within individuals (as one individual can be re-sampled as control).

	Parameter estimate	SE coefficient	Robust SE coefficient	P-value	Hazard Ratio (HR) (95% CI for HR)
fully_vaccinated_blTRUE	0.012	0.212	0.220	0.956	1.012 (0.576, 1.449)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)BP-BP	0.182	0.232	0.226	0.420	1.2 (0.668, 1.732)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)JJ	-0.236	0.425	0.421	0.576	0.79 (0.137, 1.442)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)MD-MD	-0.241	0.285	0.280	0.390	0.786 (0.354, 1.218)

The overall significance of the model is tested.

	Test statistic	Df	P-value
Likelihood ratio test	6.037588	4	0.1963587
Wald test	5.620000	4	0.2291565
Score (logrank) test	6.029537	4	0.1969533
Robust score test	5.777951	4	0.2163561

Vaccination_schedule	Arm	Measure	Estimate	SE	CI.lower	CI.upper
BP-BP	fully_vaccinated_bl==FALSE	RMST	328.928	2.429	324.166	333.690
BP-BP	fully_vaccinated_bl==TRUE	RMST	324.322	2.463	319.495	329.149
BP-BP	fully_vaccinated_bl==FALSE	RMTL	36.072	2.429	31.310	40.834
BP-BP	fully_vaccinated_bl==TRUE	RMTL	40.678	2.463	35.851	45.505
MD-MD	fully_vaccinated_bl==FALSE	RMST	316.357	5.728	305.131	327.584
MD-MD	fully_vaccinated_bl==TRUE	RMST	325.702	5.217	315.477	335.927
MD-MD	fully_vaccinated_bl==FALSE	RMTL	48.643	5.728	37.416	59.869
MD-MD	fully_vaccinated_bl==TRUE	RMTL	39.298	5.217	29.073	49.523
AZ-AZ	fully_vaccinated_bl==FALSE	RMST	322.389	6.010	310.609	334.169
AZ-AZ	fully_vaccinated_bl==TRUE	RMST	319.210	6.266	306.930	331.490
AZ-AZ	fully_vaccinated_bl==FALSE	RMTL	42.611	6.010	30.831	54.391
AZ-AZ	fully_vaccinated_bl==TRUE	RMTL	45.790	6.266	33.510	58.070
JJ	fully_vaccinated_bl==FALSE	RMST	330.197	8.954	312.647	347.746
JJ	fully_vaccinated_bl==TRUE	RMST	330.886	9.081	313.087	348.685
JJ	fully_vaccinated_bl==FALSE	RMTL	34.803	8.954	17.254	52.353
JJ	fully_vaccinated_bl==TRUE	RMTL	34.114	9.081	16.315	51.913

Vaccination_schedule	Measure	Estimate	CI.lower	CI.upper	p_value
BP-BP	RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE)	-4.606	-11.386	2.175	0.183
BP-BP	RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE)	0.986	0.966	1.007	0.183
BP-BP	RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE)	1.128	0.944	1.347	0.185
MD-MD	RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE)	9.345	-5.840	24.530	0.228
MD-MD	RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE)	1.030	0.982	1.079	0.228
MD-MD	RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE)	0.808	0.571	1.144	0.229
AZ-AZ	RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE)	-3.179	-20.196	13.838	0.714
AZ-AZ	RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE)	0.990	0.939	1.044	0.714
AZ-AZ	RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE)	1.075	0.731	1.580	0.714
JJ	RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE)	0.690	-24.306	25.685	0.957
JJ	RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE)	1.002	0.929	1.081	0.957
JJ	RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE)	0.980	0.474	2.025	0.957